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A  PREDICTION  INTERVAL  FOR  A  FIRST 
ORDER  GAUSSIAN  MARKOV  PROCESS 
by 
Toke  Jayachandran  and  T.S.  Murthy 

Let  x  (t  =  1,2,...)  be  a  stationary  Gaussian  Markov  process  of  order  one 

2  k 
with  E(x  )  =  y  and  Cov(x  ,x   )  -op.      We  derive  a  prediction  interval  for 

t  t    L  i  K 

x„    based  on  the  preceding  2n  observations  x.x„,...,x„  . 
Zn+1  1  I  2n 

1 .   INTRODUCTION 

Consider  a  stationary  Gaussian  Markov  process  of  order  one  with  E(x  )  =  y 

2  k 
and  Cov(x  ,x  ,  )  =  a  p  .   For  y  =  0  such  a  process  can  be  generated  from  an 

C    C"t"K. 

autoregressive  model 

x  =  px  .  +  e      t  -  1,2,  .  . .  (1.1) 

t     t-1    t 

with  {e  }  a  sequence  of  independent  and  identically  distributed  random  variables 

2    ,  , 
with  normal  distributions  N(0,a  ),  | p [ <1  and  x  =0.   The  process  has  many 

applications  such  as  in  modelling  certain  economic  and  meteorological  time 

series.   From  a  set  of  sample  observations  x  ,x  , ...,x   ,  in  this  paper,  we 

construct  a  conditional  prediction  interval  for  x_,  , ,  treating  one  half  of 

2k+l 

the  observations  as  conditioning  variables.   The  effect  of  the  parameters 

2 
a  ,p,k  and  the  prediction  coefficient  a  on  the  prediction  interval  is  also 

investigated. 

2.   DERIVATION  OF  PREDICTION  INTERVAL 
For  the  stochastic  process  defined  above  it  can  be  shown  [1]  that  when 
•x.  ,  k  =  l,2,...,n+l  are  fixed,  x   ,  k  =  l,2,...,n  are  conditionally  inde- 

pendent and  are  normally  distributed  with  mean  y.   =  a  +  bx'   and  variance 


2 
a   where 
o 

a  =  m(l  -  p)2/(l  +  p2) 

b  =  2P/(1  +  p2) 

X'k  =  (x2k~l  +  X2k+1)/2 

oq2  =   a2(l  -  p2)/(l  +  p2). 

Conditionally,  it  may  therefore  be  assumed  that  the  x   satisfy  the  simple 
linear  regression  model 


(2.1) 


x2k  =  a  +  bx'k  +  ek    k  -  1,2,. . . ,n 


2 
where  {e  }  are  i*i«d  N(0,a   ). 
k  o 

Given  the  sample  observations  x, ,x_,...,x_  ,x0   . ,  from  standard  regression 

1  I  zn   Zn+1 

2 
theory  the  parameters  a,b  and  a   in  (2.1)  can  be  estimated  using  the  first 

2n-l  observations  as 

b  =  s   /s 

xy  xx 

a  =  x2  -  bxT  (2.2) 

a  2  =  (s   -  b2s   )/(n-3) 
o      yy      xx 


where 


n_1  2 

x'  =   I      x'  /(n-1)-  (x  +  2x  +  2x    +  2x     +  x.   .)/Z 
.  _,    tc  1     J     D       2n-3    2n-l 

n-1 
x  =   I      x   /(n-1) 
Z    k=l   ZK 


n-1 

I 
k=l 


s,2 
xx 


x/fc  -  (n-l)x' 


a-1  2        -  2 
s   =   Ex-.-  (n-l)x 

yy   k=1  2k      2 


n-l 
Sxy  =  kl±   X'kX2k  -  Cn-l)x'x2  . 


If  x_   is  the  least  squares  predictor  of  x„   i.e.,  x.   =  a  +  bx'    then 
2n  2n        2n         n 


X2n  "  X2n 


T7^2 


,    (x'   -  x') 

i  +-iT  +  — a 

n-l     s 

xx 


has  a  student's  t-distribution  with  n-3  degrees  of  freedom;  hence 


P  [     x-   -  x.    <  t  a 
)   2n    2n'      o 


,     (x'   -  x') 
1  +-^r+    n 


rn2^ 


n-l 


xx 


=  1  -  a 


where  t  is  the  100(1  -  y)  th  percentage  point  of  the  studentls  t-distribution 

with  n-3  degrees  of  freedom.   The  above  probability  statement  can  be  converted 

into  a  prediction  interval  for  x„  ,  ..  ,  by  noting  that  x0   is  a  function  of 

Zn+i  zn 

x'   =  (x„   ,  +  x„  ,,)/2,  as  shown  below, 
n     2n-l    2n+l 

Squaring  the  inequality  and  rearranging  terms  the  above  probability 
statement  can  be  expressed  as 


f  2      2\ 

b2  -  — j  (x'   -  x')2  -  2b (x,   -x-)(x'   -x') 

i      s    /    n  2n    2    n 


xx 


2   2 


7   nt  a 

+   (x-  -  x„)Z  -  ^-     <       0 

2n    2      n-l 


=  1  -  a 


(2.3) 


or 


A(xf   -  x1)   +  B(x'   -  x')  +  C  <  0 
n  n 


=  1  -  a 


(2.4) 


where 


A  = 


2~  2 

b2 2_ 

s 

XX 


B  =  -2b(x2n  -  x2) 

2"   2 
_  2    nt  a 

C  =  (x«   -  x  )   -  ~— 

2n    2      n-1 


(2, 


A  prediction  "interval"  for  x'   =  (x„   ..  +  x„  ,  ,)/2  and  in  turn  for 

n     2n-l    2n+l 

x„   1  is  now  obtainable  in  terms  of  the  roots  of  the  quadratic  expression 

2 
in  (2.4).  If  B  -  4AC<0  i.e.,  the  roots  are  complex  the  prediction  interval 

will  be  taken  to  be  (-00,00);  in  the  other  cases  the  "interval"  can  turn  out  to 

be  a  two  sided  interval,  a  one  sided  interval  or  the  union  of  two  one  sided 

intervals.   The  different  possibilities  will  now  be  examined  in  detail. 

3.   PROPERTIES  OF  THE  PREDICTION  INTERVAL 


Case  1:   A  >  0 


Let 


b2s 


F  = 


xx 


2-   2 
t  a 
o 


Then,   A  >  0  4  F  >  1  . 


Also,  B  -  4AO0  £=> 


F  >  1  - 


_  2 
(n-1) (x   -  x2) 

2^~2 
nt  a 
o 


Hence,  A>0  =>B  -  4AC>0  and  the  prediction  interval  for  x„  ...  will  be 

2n+l 

of  the  form  (D-E,  D+E)  where 


D  =  2x'  -  x.    +  2b(x.   -  x0)/(b2  -  t2a  2/s   ) 
Zn-1       In  2  o   xx 


E  =  2ta 


t  -  \2       /      2'  2 

(x   -  x  )        /     t  a 

_2n f_  +  _n/  b2  _  o_ 

s         n-1  \      s 


xx 


xx 


(3.] 


Case  2:   A  <  0,   B  -  4AO0 
A  <  0  and  B  -  4AC>0<SV 

,    n-1    In  I  _   , 

1 ^r-^ <  F  <  1 

n       2   2 
t  a 
o 

and  the  prediction  interval  will  be  the  union  of  two  non-overlapping  intervals 

(-»,  D+E)  and  (D-E,°°).   Note  that  in  this  case  E  <  0. 

2 
Case  3:   A  f   0,  B  -  4AC<0 

As  indicated  earlier,  the  roots  will  be  complex  and  the  prediction 

interval  is  defined  to  be  (-  °° °°)  .   We  will  call  the  prediction  intervals 

resulting  from  the  above  three  cases  a  type  1,  type  2  and  a  type  3  interval, 

2 
respectively.   There  are  two  other  cases  viz.,  A  =  0  and  B  -  4AC  =  0  and 

we  have  ignored  these  possibilities  since  their  probability  of  occurrence 
is  zero.   It  should  be  clear  that  the  following  identity  holds  for  the  pre- 
diction coefficient  1  -  a. 

3 

Z        P[an  interval  of  type  i  is  obtained].   P[the  interval  will  contain 
i=l 


X2n+1]  =  1  ~  a- 

2 
To  study  the  effect  of  the  parameters  n,p,a  and  a  on  the  probability  of 

occurrence  of  the  different  types  of  intervals  we  conducted  a  simulation.  For 
each  choice  of  the  parameter  values,  2n+l  samples  are  generated  from  the  auto- 
regressive  process  (1.1)  and  the  prediction  interval  for  x.    is  calculated 

2n+l 

using  the  first  2n  values.   We  then  calculated  the  empirical  frequencies  of 
the  three  types  of  intervals,  in  1000  replications,  and  also  for  each  type  of 
interval  the  frequency  of  inclusion  of  x„   _ .   In  Table  1  we  present  the 


results  for  a  =  .05,  a  =  1.0,  n  =  14,22,30,38  as  p  takes  on  the  values 
.1,  .3,  .5,  .7,  .9.   Table  2  shows  the  effect  of  increasing  n  as  the  other 
parameters  are  held  fixed.   In  Table  3  the  standard  deviation  a   is  varied 
from  1  to  5  while  the  other  parameter  values  are  fixed.   Some  of  the  results 
are  also  presented  in  graphical  form  in  figures  1-5. 

The  following  general  conclusions  can  be  drawn  from  the  results  of  the 
simulation.   The  probability  of  obtaining  a  type  1  interval  increases  with 
p,  n  and  a.   For  n  >  15  (30  or  more  samples)  and  p  >  .5  the  probability  of 
a  type  1  interval  is  of  the  order  of  .85.   The  standard  deviation  a   does 
not  appear  to  have  any  effect  on  this  probability. 

4.   AN  EXAMPLE 

The  following  data  represents  the  monthly  Dow- Jones  industrial  averages 
for  the  years  1966-67. 

1966  1967 

Jan  31  983.51  Jan  31  879.87 

Feb  28  951.89  Feb  28  839.37 

Mar  31  924.77  Mar  31  865.98 

Apr  30  933.68  Apr  28  897.05 

May  31  884.07  May  31  852.56 

Jun  30  870.10  Jun  30  860.26 

Jul  29  847.38  Jul  31  904.24 

Aug  31  788.41  Aug  31  901.29 

Sep  30  774.22  Sep  29  926.66 

Oct  31  807.07  Oct  31  879.74 

Nov  30  791.59  Nov  30  875.81 

Dec  30  785.69  Dec  29  905.11 

Assuming  that  the  data  is  generated  by  a  Gaussian  Markov  process  of  order  one 
(a  calculation  of  lagged  correlations  supports  the  assumption  with  p  =  .8)  we 
computed  prediction  intervals  for  March  1967,  May  1967,  July  1967,  September 


1967  and  November  1967  based  on  all  the  preceding  data  and  the  results  are 
presented  below. 


Lower 

Upper 

Length 

True 

Month 

n 

Prediction 

Prediction 

of 

Value 

7 

Limit 

Limit 

Interval 
353.52 

Mar  67 

727.11 

1080.63 

865.98 

May  67 

8 

598.39 

900.56 

302.16 

852.56 

Jul  67 

9 

708.70 

1009.10 

300.40 

904.24 

Sep  67 

10 

573.59  ' 

864.87 

291.28 

926.66 

Nov  67 

11 

633.12 

899.73 

266.61 

875.81 

All  the  intervals  except  for  September  1967  contain  the  true 
value.   As  is  to  be  expected  the  length  of  the  interval  decreases 
with  an  increase  in  sample  size. 
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PROBABILITIES  OF  OCCURRENCE  OF  PREDICTION  INTERVALS 
OF  TYPES  1,2,3  IN  1000  REPLICATIONS 

TABLE  I 

a  =  1  a  =  .05 

Empirical 
n       p      P(type  1)      P(type  2)      P(type  3)      Prediction 

Coefficient 

.1        .193  .066  .741  .956 

(.917)         (.576) 

.3        .342  .080  .578  .953 

(.915)         (.775) 

.5        .269  .098  .633  .949 

7  (.888)         (.786) 

.7        .369  .119  .512  .949 

(.902)         (.874) 

.9        .539  .128  .333  .958 

(.939)         (.930) 


.1        .492  .048           .460           .941 

(.935)  (.438) 

.3        .516  .077           .407           .939 

(.924)  (.714) 

.5        .483  .110           .407           .936 

(.909)  (.818) 

.7        .791  .071           .138           .939 

(.934)  (.873) 

.9        .902  .050           .048           .948 

(.947)  (.920) 


Table  I  (Continued) 


Empirical 
P(type  1)      P(type  2)       P(type  3)     Prediction 


15 


.106  .940 


.5        .874  .044  .082  .951 


.058  .953 


.9        .979  .008  .013  .953 


.554 

.036 

(.960) 

(.722) 

.861 

.033 

(.940) 

(.758) 

.874 

.044 

(.952) 

(.841) 

.914 

.028 

(.951) 

(.929) 

.979 

.008 

(.952) 

(1.000) 

.573 

.047 

(.932) 

(.723) 

.719 

.044 

(.947) 

(.773) 

.853 

.047 

(.943) 

(.894) 

.980 

.007 

(.948) 

(.857) 

.998 

.001 

(.947) 

(1.000) 

Coefficient 


.1        .554  .036  .410  .968 


.1        .573  .047  .380  .948 

(.932)         (.723) 

.3        .719  .044  .237  .952 


21      .5        .853  .047  .100  .946 

.7        .980  .007  .013  .948 


.9        .998  .001  .001  .947 


The  numbers  in  parentheses  are  the  probabilities  that  x?  ,_ 
is  contained  in  the  interval;  for  a  type  3  interval  this  prob- 
ability is  always  1. 


PROBABILITIES  OF  OCCURRENCE  OF  PREDICTION  INTERVALS 
OF  TYPES  1,2,3  IN  1000  REPLICATIONS 

TABLE  II 

0=1  a   =  .05 


Empirical 
P(type  1)      P(type  2)       P(type  3)     Prediction 

Coefficient 


.300  .040  .660  .944 

(.867)  (.600) 

.342  .080  .578  .953 

(.915)  (.775) 

.587  .057  .356  .951 

(.942)  (.737) 

.516  .077  .407  .939 

(.924)  (.714) 

.259  .106  .635  .954' 

(.946)  (.698) 

.861  .033  .106  .940 

(.940)  (.758) 


.232  .049           .719           .951 

(.853)  (.694) 

.269  .098           .633           .949 

(.888)  (.786) 

.540  .083           .377           .950 

(.935)  (.819) 

.483  .110           .407           .936 

(.909)  (.818) 

.561  .109           .330           .951 

(.941)  (.853) 

.874  .044           .082           .951 

(.952)  (.841) 


10 


Table  II  (Continued) 


Empirical 
n      P(type  1)      P(type  2)       P(type  3)     Prediction 


Coefficient 


.308 

.072 

(.896) 

(.861) 

.369 

.119 

(.902) 

(.874) 

.586 

.095 

(.920) 

(.884) 

.791 

.071 

(.934) 

(.873) 

.840 

.065 

(.940) 

(.908) 

.914 

.028 

(.951) 

(.929) 

.620  .958 


512  .949 


.319  .942 


.138  .939 


095  .944 


,058  .953 


11 


n=i 


P-.7 


n=12 
P-.5 


PROBABILITIES  OF  OCCURRENCE  OF  PREDICTION  INTERVALS 
OF  TYPES  1,2,3  IN  1000  REPLICATIONS 

TABLE  III 

a  =  .05 

Empirical 
a  P(type  1)      P(type  2)       P(type  3)     Prediction 

Coefficient 


512  .949 


.369 

.119 

(.902) 

(.874) 

.352 

.123 

(.901) 

(.886) 

.354 

.122 

(.898) 

(.885) 

.354 

.121 

(.898) 

(.884) 

.353 

.122 

(.898) 

(.885) 

.483 

.110 

(.909) 

(.818) 

.446 

.115 

(.901) 

(.826) 

.445 

.117 

(.899) 

(.828) 

.446 

.115 

(.899) 

(.826) 

.444 

.115 

(.901) 

(.817) 

.525  .951 

.524  .950 

.525  .950 

.525  .950 

.407  .936 

.439  .936 

.438  .935 

.439  .935 

.441  .935 


12 


Fig.    1.      n  versus   P(Intervals    of    types    1,2,3) 

p  =    .5  a  =   3.0  a  =    .05 
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Fig.  2.   n  versus  P (Intervals  of  types  1,2,3) 
p  =  .7    a   =  1.0    a  =  .05 
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Fig.  3.   p  versus  P (Intervals  of  types  1,2,3) 
n=5    a=3  a=.05 
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Fig.  4.   p  versus  P(Intervals  of  types  1,2,3) 
n  =  15    a   =  1.0    a  =  .05 
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Fig.  5.   n  versus  P(type  1  interval) 
<*  =  -02,  .05,  .10   p  =  .9   a  =  1.0 
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APPENDIX 

C  PROGRAM    TO    CALCULATE    PROBABILITY    CF    TYPE    1,TYPE    2,TY°E 

C  3,TYPE   4     INTERVALS     F^P    SIMULATED    SAMPLES. 

C  PROGRAMMER    T    S   ^URTHY         SEP     1979. 

***  *t*  *i*  *r*  "f*  1*  -V  •"»*  *(*  ^*  *<*  *>*  T""(*    *■"*  *"*  •*»*  *n  *r  *!*■   *»*  *v»  *t»  i*  *.■»  f»  -|-"  "»*  'f*  '<+    -t»  -i*  t*  -i*  *r>  **•  -^  •**"  *(*  t*  *n  *»*  'i*  *»■*!*  *c    *t   *t    "r  -r  *r  *t»   ■*?  •*"  •*(* 

DIMENSION    Z(55),X(55),V(100,55),XI(50) ,YI( 50),S(55)  , 

1IC  (  5),  IP  (  5),  IPC(  5)  ,  STAT{10,7)  ,  VS(  1C  ,  10,  7)  ,  I  VS  ( 1  0) 
CALL    0VFL0W 
INDEX  =1 
SIGMA=1.0 

1  READ(5,2)     K,T 
WRI^E    (6f2)K,T 

2  FDR  ^AT{  IX,  12,  2X  =F  5.3) 
IF( K    .EQ.     0    )    GO    TO    460 
ROW =0.10 

DO    300    13=1, 5 

DO    10    J=l ,5 

3  =  (1-R0W**2     )**.5 
C  SIMULATION    OF    SAMPLES 

ISEED=12345 
10  IPCU)=0 

DO    250    IA=1, 10 

D3    5C   M=l,l00 

CALL    SNORM(ISEED,Z,K) 

X(1)=SIGMA*Z(1) 

CO    30    J=2,K 
30  X(  J  )=D0W*X(  J-  l)  +  3  *S  IGMA*Zl  J) 

DO     40    L=l  ,K 
40  V(M,L)=X(L) 

50  CONTINUE 

D3    60    II =1,5 
6  0  I  R  ( 1 1  )  =0 

DO     200    1=1,  IOC 

DO     70    J  =  l  ,K 
70  S(J)=V(I,J) 

K=K-5 

KK=K/2 
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03  6C  L=1,KK 
80  YI  (U  =S  (2*L) 

N=KK-1 

03  90  LL=1,N 
90  XI  (  LL)  =<  SI  2*  LL-1)+S(2*LL+1)  )/2.J 

XSUN=0.0 

YSUM=0.0 

sxx=c.o 

SXY=0.0 

SYY=C0 

03     ICO    KL  =  1, 5 
100  IC(KU=0 

03     110   M=1,N 

YSUV=YSUM+YI (V) 

XSUM=XSUM+XI(MJ 
110  CONTINUE 

XB=XSUM/N 

Y3=YSUM/N 

03     120    M=1,N 

SXX=SXX+(  XI  (M)-XB)**2 

SYY  =  SYY+(YI  (M)-Y3)**2 

SX*=SXY+(  XI(NI)-X9)*(YI{  MJ-YB) 
120  CONTINUE 

VRES  =  (SYY-{  (SXY**2     J/SX  X  )  )/  (N-2  ) 

EH=SXY/SXX 

AH  =  Y8-BH*XB 

SS=(T**2     )*VRES 

*=(EH**2     J-SS/SKX 

P=S(2*KK)-YB 

B=-(  2*BH*P) 

C=(°**2)-(SS*(N+1  )  J/N 

F=(BH**2     )*SXX/SS 

SSQ*1.0-N*(  P**2J /( SS*(N+1)J 

IF     (F    .L"  .    SS  C)    GH    tq    150 

IF(F    .EO.1.0  )    G3     tq       500 

0=(  2.*XB  J-S(K-l)+2.  *8H*P/A 
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130 


15  0 
160 
170 

200 

220 
250 

C 


275 

280 


285 
290 


E-(  2./A)*((  { SS*(  P**2)  /SXX)+(N  +  1)*A*SS/N)**.5) 

PIL=D-E 
PIR=D+E 
°VAL=S(  K+l) 

IF(F    .GT  .    1.0)    5  0    TC    130 
IC(2)=IC(2J+1 

IF(PVAL.lE.PIR    .OR.     PVAL.GE.PIL)     I C ( 5 ) =IC (5 ) + 1 
Go    TQ    160 
IC  (1  )=IC<1  J+l 

IF(PIL.LE.PVAL.AND.PIR.GE.PVAL)     I C L 4)=IC ( 4  J +1 
GO    TQ    160 
ICO  J=IC(3)+1 
01    17G   J=I,  5 
IR(  J)=IR(  J)+IC<JJ 
K  =  K  +  5 
CONTINUE 
CO    220    J  =  l,5 
I?C( J)=IPC( J)+IR( J) 
CONTINUE 
PRINT    STATISTICS 
STAT(  IB, 1)=R0V 
STAT( IB ,2)=I PC(1J /1CC0. C 
STA~(IB,3)=IPC(2  J/1000. 0 
STAT(  IB, 4  )=  IPC(3  J/1000. 0 
IF(STAT(  IB, 2)  .EQ.O.C)    GO    TO    275 
STAT( 13  ,5) -I  PC (4)  /( STAT  (I B, 2  J *1 000 .0  J 
GO    TC   280 
ST4T  (  IB, 5  J=  I°C(4  J 

IF( ST*T( IB, 3)     «EQ.     0.0)    GO    TQ    285 
STATU8,6)=I  PC  (5)  /(  STAT  ( I  B,  3  J  *1  000  .0  ) 
GC    rC    290 
ST  AT(IB,6  )=IPC(5  J 

SIG=STAT(  IB  ,2)*STAT(  IB  ,5)  + STAT  (I  B,3)*STAT(  IB,  6) 
1    *STAT(  IB  ,4) 
S-£T(IB,7  J=S  IG 
RTW=ROW+0  .2 
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300         CONTINUE 

DO    305    IQ=1 ,5 

CO    305    J0=lt7 

VS<  INDEX,  10  ,JO)  =  STAT(  IQ,JQJ 
305         CONTINUE 

IVS( INOEX)=N 

INDEX=INDEX-H 

ISZ=K-5 

WRITE(6,310)  ISZ.  -SIGMA, M 

UR  ITE    (  6,325) 

WRI TE( 6,3  50)  (  (STAT< K,L)   ,L=I ,7)  ,K  =  L  ,5) 
310         FORNATClXt1        SAMPLE    SIZE   =    ',15,'    SIGMA    =     SF5.0,        ■ 

IN    =    •  ,  15,  /) 
325         F3RMAT(1X,'  10W  TYPE    1  TYFE    2  TY^E    3 

1FR0B.1  PRCB.2  CON. PEG         ',/,70(,-,J) 

350  FOP v/iT  (       7(F3  .3,  2X  ),/  ) 

WRITE(6,485) 

GO    TO    1 
460         CCRF=0.1 

INCEX=  INDEX-1 

00     470   J  =  l,5 

fcRITE    (6,472)    CORR*SIGWA 

WRITE(6 ,475  ) 
DC     471    1=1,  INDEX 

WU  7E( 6,4  80) (  IVSCI )  ,  (VS  (I  ,J  ,K)  ,K  =  2,7) ) 

471  CONTiNUE 
C0RF=CORR+O  .2 

470         CONTINUE 

472  FORMAT(        •     CORR.COEFF.     =    •  , F5  .3 , '     SIGMA    =    •,F5.0,//) 
475  FORVATC     SAMPLE    SIZE  TYPE   1  TY°E    2  TY°E    3  °R0 

1E.1  DRnn.2         CON. PEG    ',/,70('-')) 

480         FORMATU  5  ,  5X  ,  6(  F  8.  3  , 2  X)  ,/) 
500  STQF 

EMC 
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